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Abstract 



Embedding microscopic sensors, computers and actuators into materials 
allows physical systems to actively monitor and respond to their environments. 
This leads to the possibility of creating smart matter, i.e., materials whose 
f-- properties can be changed under program control to suit varying constraints. 

A key difficulty in realizing the potential of smart matter is developing the 
. appropriate control programs. We present a market-based multiagent solution 

| to the problem of maintaining a physical system near an unstable configuration, 

a particularly challenging application for smart matter. This market control 
leads to stability by focussing control forces in those parts of the system where 
they are most needed. Moreover, it does so even when some actuators fail to 
work and without requiring the agents to have a detailed model of the physical 
£H ■ system. 

o ' 
o 

1 Introduction 

X 

Embedding microscopic sensors, computers and actuators into materials allows phys- 
ical systems to actively monitor and respond to their environments in precisely con- 
trolled ways. This is particularly so for microelectromechanical systems (MEMS) |IJ 
H where the devices are fabricated together in single silicon wafers. Applications 
include environmental monitors, drag reduction in fluid flow, compact data storage 
and improved material properties. 

In many such applications the relevant mechanical processes are slow compared to 
sensor, computation and communication speeds. This gives a smart matter regime, 
where control programs execute many steps within the time available for respond- 
ing to mechanical changes. A key difficulty in realizing smart matter's potential is 
developing the control programs. This is due to the need to robustly coordinate a 
physically distributed real-time response with many elements in the face of failures, 
delays, an unpredictable environment and a limited ability to accurately model the 
system's behavior. This is especially true in the mass production of smart materials 
where manufacturing tolerances and occasional defects will cause the physical system 
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to differ somewhat from its nominal specification. These characteristics limit the 
effectiveness of conventional control algorithms, which rely on a single global proces- 
sor with rapid access to the full state of the system and detailed knowledge of its 
behavior. 

A more robust approach for such systems uses a collection of autonomous agents, 
that each deal with a limited part of the overall control problem. Individual agents 
can be associated with each sensor or actuator in the material, or with various ag- 
gregations of these devices, to provide a mapping between agents and physical loca- 
tion. This leads to a community of computational agents which, in their interactions, 



strategies, and competition for resources, resemble natural ecosystems ||13fl . Dis- 
tributed controls allow the system as a whole to adapt to changes in the environment 
or disturbances to individual components [JTO] • 

Multiagent systems have been extensively studied in the context of distributed 
problem solving j|, |7|, |T5|] . They have also been applied to problems involved in 
acting in the physical world, such as distributed traffic control [|17]], flexible manufac- 
turing [J22[|, the design of robotic systems |TR EH], and self-assembly of structures [EO 



However, the use of multiagent systems for controlling smart matter is a challenging 
new application due to the very tight coupling between the computational agents 
and their embedding in physical space. Specifically, in addition to computational 
interactions between agents from the exchange of information, there are mechanical 
interactions whose strength decreases with the physical distance between them. 

In this paper we present a novel control strategy for unstable dynamical systems 
based on market mechanisms. This is a particularly challenging problem, for in the 
absence of controls, the physics of an unstable system will drive it rapidly away from 
the desired configuration. This is the case, for example, for a structural beam whose 
load is large enough to cause it to buckle and break. In such cases, weak control forces, 
if applied properly, can counter departures from the unstable configuration while they 
are still small. Successful control leads to a virtual strengthening and stiffening of the 
material. Intentionally removing this control also allows for very rapid changes of the 
system into other desired configurations. Thus an effective way of controlling unstable 
systems opens up novel possibilities for making structures extremely adaptive. 



2 Dynamics of Unstable Smart Matter 

The devices embedded in smart matter are associated with computational agents 
that use the sensor information to determine appropriate actuator forces. The overall 
system dynamics is a combination of the behavior at the location of these agents and 
the behavior of the material between the agent locations. In mechanical systems, 
displacements associated with short length scales involve relatively large restoring 
forces, high frequency oscillations and rapid damping. Hence, they are not important 
for the overall stability |TTJ. Instead, stability is primarily determined by the lowest 



frequency modes. We assume that there are enough agents so that their typical 
spacing is much smaller than the wavelengths associated with these lowest modes. 
Hence, the lower frequency dynamics is sufficiently characterized by the displacements 
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Figure 1: An unstable dynamical system, a) The unstable chain with the mass points 
displaced from the unstable fixed point which is indicated by the horizontal dashed line. 
The masses are coupled to their neighbors with springs, and those at the end of the chain 
are connected to a rigid wall, b) A chain of upward-pointing pendulae connected by springs 
as an example of an unstable spatially extended system. 



at the locations of the agents only. The high-frequency dynamics of the physical 
substrate between agents serves only to couple the agents' displacements. 

The system we studied, illustrated in Fig. |l|a, consists of n mass points connected 
to their neighbors by springs. In addition a destabilizing force proportional to the 
displacement acts on each mass point. This force models the behavior of unstable 
fixed points: the force is zero exactly at the fixed point, but acts to amplify any 
small deviations away from the fixed point. This system can be construed as a linear 
approximation to the behavior of a variety of dynamical systems near an unstable 
fixed point, such as the inverted pendulae shown in the Fig. |l|b. In the absence of 
control, any small initial displacement away from the vertical position rapidly leads to 
all the masses falling over. In this case, the lowest mode consists of all the pendulae 
falling over in the same direction and is the most rapidly unstable mode of behavior 
for this system. By contrast, higher modes, operating at shorter length scales, consist 
of the masses falling in different directions so that springs between them act to reduce 
the rate of falling. 

The system's physical behavior is described by 

1. the number of mass points n 

2. the spring constant k of the springs 

3. a destabilizing force coefficient / 

4. a damping force coefficient g 

We also suppose the mass of each point is equal to one. The resulting dynamics of 
the unstable chain is given byQ [§]: 

—r- = Vi 

— = k(xi„i-Xi) + k(x i+ i-Xi) + fxi - gvi + H { 

1 We used a standard ordinary-differential-equation solver |b| to determine the controlled system's 
behaviors. 
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where Xi is the displacement of mass point i, Vi is the corresponding velocity, and 
Xq = x n+ i = is the boundary condition. The term in Eq. (Q) is the additional 
control force produced by the actuator attached to mass point i. We suppose the 
magnitude of this control force is proportional to the power Pi used by the actuator. 
For reasons of simplicity we use a proportionality factor of 1. 

For these systems, the long time response to any initial condition is determined 
by the eigenvalues of the matrix corresponding to the right hand side of Eq. (|l|). 
Specifically, if the control force makes all eigenvalues have negative real parts, the 
system is stable ||11|| . The corresponding eigenvectors are the system's modes. Thus 
to evaluate stability for all initial conditions, we can use any single initial condition 
that includes contributions from all modes. If there are any unstable modes, the 
displacements will then grow. We used this technique to evaluate stability in the 
experiments described below. 



3 A Power Market for Control 

The control problem is how hard to push on the various mass points to maintain 
them at the unstable fixed point. This problem can involve various goals, such as 
maintaining stability in spite of perturbations typically delivered by the system's 
environment, using only weak control forces so the actuators are easy and cheap to 
fabricate, continuing to operate even with sensor noise and actuator failures, and 
being simple to program, e.g., by not requiring a detailed physical model of the 
system. 

Computational markets are one approach to this control problem || |TJ|, |TJ, 



|16| , |21| , |23| , |24|| . As in economics, the use of prices provides a flexible mechanism for 
allocating resources, with relatively low information requirements |J: a single price 
summarizes the current demand for each resource. 

In designing a market of computational agents, a key issue is to identify the con- 
sumers and producers of the goods to be traded. Various preferences and constraints 
are introduced through the definition of the agents' utilities. This ability to explicitly 
program utility functions is an important difference from the situation with human 
markets. Finally, the market mechanism for matching buyers and sellers must be 
specified. 

In the market control of smart matter treated here, actuators, or the corresponding 
mass points to which they are attached, are treated as consumers. The external power 
sources are the producers and as such are separate from consumers. All consumers 
start with a specified amount of money. All the profit that the producers get from 
selling power to consumers is equally redistributed to the consumers. This funding 
policy implies that the total amount of money in the system will stay constant. 

In the spirit of the smart matter regime, where control computations are fast 
compared to the relevant mechanical time scales, we assume a market mechanism 
that rapidly finds the equilibrium point where overall supply and demand are equal. 
Possible mechanisms include a centralized auction or decentralized bilateral trades or 
arbitrage. This equilibrium determines the price and the amount of power traded. 
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Each actuator gets the amount of power that it offers to buy for the equilibrium price 
and uses this power to push the unstable chain. 

The utility function for using power P reflects a trade-off between using power to 
act against a displacement and the loss of wealth involved. While a variety of utility 
functions are possible, a particularly simple one for agent i, expressed in terms of the 
price of the power, p, and the agent's wealth, wt, is: 

Ui = - 7 ?- P P 2 + bP\X i \ (2) 
ZWi 

where 

n 

X i = Yl a iJ X J (3) 
3=1 

is a linear combination of the displacements of all mass points that provides informa- 
tion about the chain's state. The parameter b determines the relative importance to 
an agent of responding to displacements compared to conserving its wealth for future 
use. 

Actuator i always pushes in the opposite direction of Xi, i.e., it acts to reduce 
the value of X«. In this paper we focus on the simple case of purely local control 
where = 1 when i — j and is otherwise. Thus, consumer % considers only its 
own displacement X; L . For simplicity, we use an ideal competitive market in which 
each consumer and producer acts as though its individual choice has no affect on 
the overall price, and agents do not account for the redistribution of profits via the 
funding policy. Thus a consumer's demand function is obtained by maximizing its 
utility function as a function of power: 

^ = -P- + bW = =► P.ip) = b\X^ (4) 
dP Wi p 

This demand function causes the agent to demand more power when the displacement 
it tries to control is large. It also reflects the trade-off in maintaining wealth: de- 
mand decreases with increasing price and when agents have little wealth. The overall 
demand function for the system is just the sum of these individual demands, giving 

p dcmand (p) = -El^k (5) 
p — 

Similarly, each producer tries to maximize its profit p given by the difference 
between its revenue from selling power and its production cost C(P): p = pP — C(P). 
To provide a constraint on the system to minimize the power use, we select a cost 
function for which the cost per unit of power, C{P)/P increases with the amount of 
power. A simple example of such a cost function is 

C(P) = ^- a p2 ( 6 ) 

The parameter a reflects the relative importance of conserving power and maintaining 
stability. We obtain the producer's supply function by maximizing its profit: 
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This is the same for all producers, so the overall supply function is then just 



psupply^\ 



nap 



From this the price and amount of traded power is determined by the point where 
the overall supply and demand curves intersect, i.e., p demand (p) = p su PP 1 y(p). For our 
choices of the utility and cost functions, this condition can be solved analytically to 
give 



Ptrade 



\ 



-El* (9) 



Given this equilibrium price, agent i then gets an amount of power equal to Piijptrade) 
according to Eq. (f|) and the resulting control force is directly proportional to received 
power. 

We can also consider the case where the amount of power available to the system 
is limited to P^^ al - This hard constraint has the effect of limiting the overall supply 
function when the price is high so it becomes 

psuppiy/ n \ = / na P Hp<P^ al /na . . 

l^Sax al otherwise 1 ; 

The final aspect of the market dynamics is how the wealth changes with time. 
This is given by 

^ = -pPi(p) + IpP d — d ( p ) 

CaAj Ti / -I -I \ 

b , ( U ) 



-b\X i \w i + -Y,\X; 



n j=1 



because we use the funding policy that all expenditures are returned equally to the 
agents in the system. 



4 Comparing with Local Controls 

As a simple comparison for the market behavior, we also study a local control method. 
In this case, each actuator i pushes with a strength that is proportional to the dis- 
placement of its respective mass point, and ignores the displacements of all other 
mass points. Specifically, the local controls are simply given by Hi = —cx{. Other 
control strategies JTTJ] can try to estimate the amplitude of the lowest modes and push 
only against these modes, since these are the ones most important for stability. 

For comparison with the market, we restrict ourselves to the case where the 
amount of available power is limited. This is useful for evaluating the ability of 
different control methods to maintain stability using only weak forces. We distin- 
guish two ways the power could be limited for the local control. In the first, each 
actuator is separately limited to use no more than P max power (local control 1), which 
corresponds to a situation where each actuator has a separate power source such as 
its own battery. Any actuator that requests more power than this maximum has its 
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control force reduced to require only P max , i-e., \Hi\ = P max . The second local control 
allows available power to be moved among the different actuators and is limited only 
in that all actuators together cannot use more than P^ax^ = ^-Pnax power (local 
control 2). This overall limit is implemented by comparing the total power requested 
according to the local control, i.e., P re quest = cJ2i to the maximum available. If 
the requested amount exceeds the maximum, each agent has its power reduced by 
the factor P^°^ a1 / P req uest so that the overall used power equals the global limit. The 
corresponding market has a total available power of Pm ax al = n -Pmax- 



5 Results 

We studied a chain composed of 27 mass points, all of them having unit mass and 
connected by springs with a spring constant of value 1 and damping coefficient 0.1. 
The destabilizing force coefficient is 0.2, which is sufficient to make the system unsta- 
ble when there is no control force. All agents start with an initial wealth of 50 money 
units and we are using the values a = 0.05 and b = 0.001 in the cost and utility 
functions. For definiteness, we chose an initial condition where the single element in 
the middle of the chain had a unit displacement and all other values were at zero. 
This configuration includes a contribution from all the modes of the system, which 
are just sinusoidal waves in this chain with uniform masses and spring constants ||. 
For the local control, we used c = 0.2, which is more than sufficient to ensure stable 
control when power is unlimited [O] . 



With P max = 0.012, Fig. |2] compares the performances of both local and market 
controls. We show both the total power use J2i Pi an d the average displacement of 
the chain J2i \ x i\/ n - As can be seen, for the chosen parameter values the market is 
able to control the unstable chain in spite of the fact that the power is limited to 
a global maximum. This limit is reached several times. The local controls (1 and 
2), on the other hand, fail in both cases, as seen in the figure. These results were 
obtained in a simulation run that lasted 20 time units. A longer simulation shows 
that the overall power usage and average displacements decrease with time for the 
market control while displacement continues increasing for the local controls. 

Since the power cost function C(P) does not change, the overall supply curve never 
changes, as shown in Fig. |] which displays the supply curve and some demand curves 
for different times. The demand curves depend on the displacements and wealth of 
the agents. Since these are dynamical variables, overall demand curve changes in 
time. In addition to the times I, II and III marked in Fig. ^a, we also plot the overall 
demand curves for later times IV, V and VI. This shows that the amount of traded 
power decreases with time while the unstable chain is controlled by the market. 

To demonstrate how robust the market mechanism is, we show in Fig. |] the 
system's response when an actuator breaks down. In this case we slightly increase the 
amount of power available compared to the simulation used for Fig. |2| to P max = 0.015, 
so that the local controller 2 can also control the unbroken system. With the system 
initially functioning properly, we turned off the actuator in the middle of the chain 
after 10 time units and observed the consequent evolution. As can be seen, the market 
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Figure 2: a) Time development of the overall power usage for a market control (solid) and 
local controls 1 and 2 (dashed) in the case of limited available power. With the same power 
limit in all three cases, the market is the only one that can control the unstable chain. 
Points I, II and III mark the times at which the supply and demand curves intersections 
are shown in Fig. ||[ b) Corresponding time development of the average displacement for a 
market control and local controls 1 and 2. The market reduces the average displacement 
with time whereas the local control is not able to prevent it from growing. 
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Figure 3: Overall supply curve (dashed) and the overall demand curves (solid) at times 
1=1.6, 11=8.0, 111=18.4, IV=40.0, V=60.0 and VI=90.0 for the market example of Fig. |. 



is still able to control the system whereas the local control fails to do so. 



6 Discussion 

In this paper we presented a novel mechanism of controlling unstable dynamical 
systems by means of a multiagent system using a market mechanism. We described 
how we defined consumers' and producers' utility functions that lead to the overall 
supply and demand curves and evaluated the price and amount of traded power within 
the system. 

We showed that the market approach is able to control an unstable dynamical 
system in the case of limited power whereas a traditional local control strategy fails 
under the same assumptions. We also demonstrated that a market control adapts 
better to cases when an actuator breaks during the controlling process. These results 
show that a market control can be more robust than a local control when operating 
with given power constraints by focussing the power in those parts of the system where 
it is most needed. This not only reduces total power use but, more importantly, also 
allows control with weaker, and thus easier to fabricate, actuators. 

The power of market approaches to control lies in the fact that relatively little 
knowledge of the system to be controlled is needed. This is in stark contrast to tradi- 
tional AI approaches, which use symbolic reasoning with extremely detailed models 
of the physical system. However, while providing a very robust and simple design 
methodology, a market approach suffers from the lack of a high level explanation for 
its global behavior. An interesting open issue is to combine this approach with the 
more traditional AI one. 

Although we have chosen particular forms of utility, supply and demand func- 
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Figure 4: Comparison of a market control and a local control in the case where one actuator 
breaks after 10 time units. Both control strategies would be able to control the system when 
all actuators would work perfectly. The market is still able to control the system although 
one actuator is broken but the local controller fails, a) Overall used power vs. time, b) 
Average displacement vs. time. 
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tions, there are many other functional forms that can also control the system. These 
could include additional goals, such as faster recovery from sudden changes and min- 
imizing the number of active actuators. Furthermore, different funding strategies are 
possible, where profits are shared unequally among agents or the funds are allocated 
by an external agent. A very promising approach is the possibility of improving the 
performance of the system by having different market organizations that change in 
time. In our system, this corresponds to the agents learning to use information on the 
displacements or velocities of their neighbors when making their control decisions. In 
this way the multiagent system would take advantage of the fact that markets are a 
simple and powerful discovery process: new methods for selecting trades can be tried 
by a few consumers or producers and, if successful relative to existing approaches, 
gradually spread to other agents. Such a learning mechanism could help the sys- 
tem discover those organizational structures that lead to improved performance and 
adaptability. 
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